Weighted Mean Squared Error Criterion with Fixed-level Modification for Linear-phase Fir Filter Design*

This paper describes a new approach--a Fixed-Level Least Squares (FLLS) method for linear-phase FIR filter design. It is intended for rejection of the Gibbs phenomenon through the introduction of a set of equally spaced fixed levels in the transition band and subsequent redefinition of the approximated and weighted functions. Detailed mathematical solutions of the problem as well as many examples are given. The results in graphical form are shown as an output of the FLLS software model. 1. Background The Least Squares (LS) method gained its practical application as an alternative to the well-known McClellan-Parks (MP) approach [8] for the design of linear-phase FIR digital filters. There are numerous publications [see reference list], in which the advantages and the shortcomings of the LS method are discussed, and new modifications of this method are proposed. For example, in [6] examination of the LS method in the time domain is done on the basis of the already-known input autocorrelation function and the crosscorrelation function between the input and the desired output. Vaidyanathan et al. [17] defined a new term eigenfilter: a filter, completely constructed according to the LS method, whose coefficients are the components of an eigenvector of a real, symmetric, and positive-definite matrix. The weighted LS approach for the design of filters with equiripple passbands and stopbands is discussed in [2]. Adams et al. [1] examined an extension to the LS method where filters with minimax passband and least-squares stopband are designed. An effective optimizational method for the design of high-order filters with discrete coefficients is given in [7], and some popular and widely used design formulas are mentioned in [5]. Tiajev [15], [16] describes a procedure for the design of LS filters with a fiat passband frequency response by the introduction of * Received November 22, 1994; accepted June 20, 1995. 1 Department of Computer-Aided Design, Higher Institute of Architecture and Civil Engineering, 1 Hr. Smirneski Blvd., 1421 Sofia, Bulgaria. 582 MOLLOVA a quadratic trinomial. Very simple from a computational viewpoint is the method of [4], which minimizes the least-square error in the passband only. The application of the LS approach for Type 1 and Type 2 filters is given in detail in [9], [10]. The basic problem with the LS method, as well as with the other approximation methods for FIR filter design, is the Gibbs phenomenon [12], which shows as a "ripple," found near the edge of the passband. It is a result of the discontinuity of the desired frequency response. According to [11], there are four approaches for reducing the overshoot (Gibbs phenomenon) occurring near a discontinuity in the LS method. The first approach involves the introduction of a function that eliminates the discontinuity in the transition band--spline function [3], straight line [11], trigonometric function, etc. In the second approach the error criterion is changed: the transition band is not put under optimization (the so-called don't care region) [4]. The third and fourth approaches use a positive weight function [11] and window functions [11], [12], [14], respectively. Here a new approach is presented that is a modification of the first and third LS approaches mentioned above. Redefinition of the approximated and the weight functions in the transition band is done with the help of the introduction of f numbers of equally spaced fixed levels in this band (FLLS method). 2. Problem formulation and solution To solve the problem we use as a starting point the well-known relationship for the least-mean square error: f0 "5 E = e(w)[A(o9) d i ~ ( o 9 , ~)]2 do9 (1) where Q(o9) is a positive weight function, A(o9) is the approximated (given) function, and o9 6 [0, 0.5] is the normalized frequency. ~(w, ?) serves as an approximation function of the given one A(w), and for a Type 1 filter it is k (09, -~) = ~ el cos 12zro9 (2) /=0 where k = (N 1)/2, and N is the order of the filter. The coefficients of the digital transfer function N--1 H(z) = Z btz-t (3) 1=0 correspond to the coefficients ct of 9 (o9, ~) as Cl bk+l = b k l = ~ , l = 1 . . . . . k (4a) b~ = Co. (4b) It is known [5], [11 ], [ 17] that the upper defined weight least-mean squared estimation for proximity between the A(og) and ~(o9, ?) functions leads to the solution MSE CRITERION FOR FIR FILTERS 583 of the set of linear equations k dn,lCl =" dn,k+l, n = 0 . . . . . k . ( 5 ) / = 0 The coefficients dn,l and the free term dn,k+l are defined by the relationships L 0 . 5 d~j = Q(co) cos(23rnw) cos(2~/co) do) (6a) L O.5 d.,~+l = Q(w)A(o ) ) cos(2~nco) do). (6b) A(w)i I a I a 2 a k af.1 af 0 Figure 1. t"l"~" 2 A p = w s5 w pb k I I .f-1 I " " f Wpb Wsb 0 .5 w The most important aspect of the presented FLLS method is the redefinition of the approximated function A(co) in the transition band and the corresponding changes in the weighted function Q(co). Thus, for example, for a lowpass filter f ( f > 1) fixed levels in the transition band are set, with the intention of eliminating the Gibbs phenomenon (Figure 1). Here copO (passband) and cosb (stopband) are normalized frequencies in the passband and the stopband of the filter, respectively. The transition band Ap is divided into ( 2 f + 1) equally spaced sublevels. The values of A(co) and Q(co) for the sublevels, shown in Figure 1 with a solid line, are given in Table 1. In the zones drawn with a dashed line in Figure 1, the approximated function A(co) remains undefined and the weight function Q(co) has a zero value. The vector fi = (al, a2 . . . . . ay) describes the fixed levels of A (co) in the transition band. It has been set that the values o f a i , i = 1, 2 . . . . . f , divide the [0, 1] interval on the A(co) axis into i + 1 equal parts. In other words, the elements of the vector are defined for a lowpass filter according to the relationship f i + l a i = , i = 1 , 2 . . . . . f . f + l For example if we have three levels in the transition band ( f = 3) the vector fi isfi [3, 2 = ~, 88 The elements of the vector 7 = (ti, t2 . . . . . t f ) from Table 1 584 MOLLOVA Table 1. Lowpass filter (LP) Highpass filter (HP) Ap = Wsb copb Ap -= cod, -Wsb 0)1 = copb'~ 0")2 : O)sb 0)I ~ &)sb'~ 092 : copb A(CO) co ~ [0, wl] 1 for LP filter 0 for HP filter r E [COl + 2-~+I Ap, COl + 2--~+I AP] ai CO E [Wl + T-~+I Ap, wl + T-~+I Ap] as w ~ [col + 2_Lz22f-2 A 1 2y+l ~P, o91 + sf+l P] af_l O) E [0.91 .-~ ~ ~-1 Ap, COl q'~ Ap] a f co ~ [cos, 0.5] 0 for LP filter 1 for HP filter Q(~o)